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Abstract 

We geometrically describe optimal control problems in terms of Morse families in the 
Hamiltonian framework. These geometric structures allow us to recover the classical first order 
necessary conditions for optimality and the starting point to run an integrability algorithm. 
Moreover the integrability algorithm is adapted to optimal control problems in such a way that 
the trajectories originated by discontinuous controls are also obtained. From the Hamiltonian 
viewpoint we obtain the equations of motion for optimal control problems in the Lagrangian 
formalism by means of a proper Lagrangian submanifold. Singular optimal control problems 
and overdetermined ones are also studied along the paper. 



1 Introduction 



In the late fifties the interest in optimal control problems grew amazingly due to the applications 
in military missions and later on due to the applications in space missions. Pontryagin and his col- 
laborators provided us with a useful result to characterize candidates to be optimal solutions [TU] . 
Since then great efforts have been made to understand optimal control problems from a differ- 
ential geometric viewpoint (see [2j El [16], [21] and references therein). It turned out that both 
symplectic [20] and presymplectic [5] formalisms were useful in that sense. 
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In this paper we present a novel formulation of optimal control problems by using different 
geometric tools (see [2j and [13] for a different but related approach in the scope of calculus of 
variations). Those tools are Morse families [T7] and Lagrangian submanifolds which are con- 
nected by the fact that any Lagrangian submanifold is locally described by a Morse family. We will 
show how these geometric elements, once properly adapted to optimal control theory, summarize 
a weaker version of the first order necessary conditions given by Pontryagin's Maximum Principle. 
At this stage a suitable adaptation of the integrability algorithm in [18] can be considered to find 
the set of candidates to be optimal solutions. Both Morse families in Section H] and Lagrangian 
submanifolds in Section describe a new setting to distinguish and discuss different kinds of opti- 
mal control problems. All these results are applied to different singular and real examples, among 
them overdetermined control systems. These last systems are very useful for some applications in 
order to gain mobility in the control system under study. 

Moreover, it is important to stress that in our formulation the controls will play the role of 
parameters, and the spaces where the associated optimal control equations are given are exactly 
the same ones as for classical Hamiltonian and Lagrangian mechanics. In this sense our point of 
view of a control problem is similar to [T6j . 

The outline of the paper is as follows. Section [2] summarizes all the geometric tools and results 
necessary for the paper. Section [3] briefly recalls the notion of optimal control problems and 
Pontryagin's Maximum Principle. Some of the main contributions of this paper are in Sections H] 
and [6] where Morse families and Lagrangian submanifolds, respectively, are used to describe optimal 
control problems and their regularity. Several examples are provided to make the theory clearer. 
Another important constribution appears in Section where the integrability algorithm in [T8] is 
adapted to optimal control problems with the purpose of recovering even the solutions originated 
by bang- bang controls. 

2 Geometric preliminaries 

In this section we briefly introduce all the definitions and results from differential geometry that 
are necessary in the sequel. See [U Q3J [T7J [23J ES] for more details. 

2.1 Lagrangian submanifolds and Morse families 

We define the two main geometric tools that are used in this paper: Lagrangian submanifolds and 
Morse families. More details can be found in [TJ [101 HH [T71 126] and references therein. 

Let us recall that a symplectic vector space is a pair (E, Q) where E is a vector space and 
Q: E x E — ^ 1 is a skew-symmetric bilinear map of maximal rank. We distinguish the following 
type of vector subspaces of a symplectic vector space: 

Definition 2.1. Let (E, Q) be a symplectic vector space and F C E be a subspace. The Q- 
orthogonal complement of F is the subspace defined by 

F L = {eeE \ Q(e, e') = for all e' E F}. 

A subspace F of a symplectic vector space is called 

(i) isotropic if F C F- 1 , that is, Q(e, e') = for all e, e' G F. 
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(ii) Lagrangian if F is isotropic and has an isotropic complement, that is, E — F © F', where F' 
is isotropic. 

A well-known characterization of Lagrangian subspaces of a finite dimensional symplectic vector 
space is the following one: 

Proposition 2.2. Let (E, Q) be a symplectic vector space and F C E a subspace. Then the 
following assertions are equivalent: 

(i) F is Lagrangian, 

(ii) F = F ± , 

(Hi) F is isotropic and dimF = |dimi?. 

As a consequence, Lagrangian subspace F of E can be characterized by checking if it has half 
the dimension of E and Q\f = 0. 

Remember that a symplectic manifold (M, u) is defined by a differentiable manifold M and a 
non-degenerate closed 2-form u on M. Therefore, for each x G M, (T X M, u x ) is a symplectic vector 
space and a symplectic manifold has even dimension. Denote by b w : TM — > T*M the isomorphism 

b u {v) = i v co, with v G TM, and by jj A/ = (b w ) _1 . 

The notion of Lagrangian subspace can be transferred to submanifolds by requesting that the 
tangent space of the submanifold is a Lagrangian subspace for every point in the submanifold of a 
symplectic manifold. 

Definition 2.3. Let (M, to) be a symplectic manifold and f : N — > M an immersion. It is said 
that N is a Lagrangian immersed submanifold of (M,u) if (T x f ) (T x iV) C Tff x \M is a Lagrangian 
subspace for each x 6 N. We say that f is a Lagrangian immersion. 

Note that f : N — > M is isotropic if and only if f*u = 0, that is, u(T x f(u x ),T x f(v x )) = for 
every u x ,v x G T X N and for every x G N. 

The canonical model of symplectic manifold is the cotangent bundle T*Q of an arbitrary man- 
ifold Q. Denote by ttq : T*Q — > Q the canonical projection and define a canonical 1-form 9q on 
T*Q by 

where X aq G T aq T*Q, a q G T*Q and q G Q. If we consider bundle coordinates (q\pi) on T*Q such 
that n Q (q\pi) = q\ then 

6 Q = pidq* . 

The 2-form uq = —d6g is a symplectic form on T*Q with local expression 

ujq = dq l A dp^ 

The Darboux theorem states that this is the local model for an arbitrary symplectic manifold 
(M,uj): there exist local coordinates {q l ,Pi) in a neighbourhood of each point in M such that 
cu = dq l A dpi. 

There are different ways to define Lagrangian submanifolds as we will show in the sequel. A 
relevant example of Lagrangian submanifold of the cotangent bundle is the following one. For 
complete proofs of the following statements we refer the reader to [T7] . 
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Proposition 2.4. Let a be a one-form on Q. The submanifold Ima is a Lagrangian submanifold 
if and only if a is closed. 

Given a symplectic manifold (M,u), dimM = 2n, it is well-known that the tangent bundle 
TM is equipped with a symplectic structure denoted by d T u, where d T u denotes the tangent 
lift of u to TM. If we take Darboux coordinates (q l ,Pi) on M, that is, u = dq l A dpi, then 
drui = dq l A dpi + dq l A dpi, where (q l ,Pi) q\Pi) are the induced coordinates on TM. Denoting the 
bundle coordinates on T*M by (q\pu a>i, b 1 ), then u M = d<f A da t + dp, A db\ If b w : TM T*M 
is the isomorphism defined by to, that is, \> u (v) = i v u, then \> ul (q\Pi,q\Pi) = (q\Pi~, —pi,q l ). 

Given a function H : M — ¥ R, and its associated Hamiltonian vector field Xh, that is, ix H u = 
dH, the image of Xh, Im X^, is a Lagrangian submanifold of (TM, dr^). Moreover, given a vector 
field X on M, it is locally Hamiltonian if and only if its image X(M) is a Lagrangian submanifold 
of (TM,d r w). It is interesting to note that d T u = -b* uj m and \> W (X H (M)) = dH(M). 

Another geometric element that can be used to define Lagrangian submanifolds is a Morse 
family. The notion of a Morse family or phase function was introduced by L. Hormander [15]. Here 
we just give the key definitions and essential results for this paper using a similar notation to the 
one in [17] . 

Definition 2.5. Let 7r : B — > Q be a submersion of a differentiable manifold B onto a different iable 
manifold Q. Let 5: B — > M, be a differentiable function. The function S" is called a Morse family if 
the image of the differential of S, dS(B) C T*I?, and the conormal bundle 

Q w = (ker Ttt)° = {a E T*B | (a,u) = 0, for all v G ker T wbM it} C T*5 

are transverse in T*5, that is, 

\/a£Q w C)dS(B)CT*B, T a (dS(B)) + T a Q n = T a {T*B). (1) 



Observe that under these transversality conditions Q n D d5"(i?) is an isotropic submanifold of 
(T*B,ub), since it is contained in the Lagrangian submanifold dS(B). Moreover 

dim(Q w n dS(B)) = dimQ. 

If the submersion 7r : i? — > Q is expressed locally by ir(q t ,y a ) = (q 1 ), then 5 : I? — )■ M is a Morse 
family in a local sense if the 

1 / d 2 S d 2 S \ 
rank or - — , , 7: — — r is maximal 

\oq i oy b oy a oy b J 

dS 

for all (V, y a ) satisfying — — = 0. 

Define the morphism j : Q T — > T*Q by 

(77, u> = 0(77), TttO)) 

for all 77 G Q,r and for all v G T n ^B. The following result will be useful in the sequel. For a proof 
see, for instance, [T71 Appendix 7, Proposition 1.12]. 

Proposition 2.6. Let S : 5 — >■ M. be a Morse family. The restriction of the morphism ) : Q n — > T*Q 
to the isotropic submanifold Q n n dS(B) is a Lagrangian immersion of Q n fl dS(B) in (T*Q,u)q). 
This Lagrangian immersion is said to be generated by the Morse family S . 
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The assumptions in Proposition 12.61 can be weakened by requiring that Q w and dS(B) are only 
weakly transverse, instead of the transversality required in the Definition 12.51 of a Morse family. 
Remember that two submanifolds are weakly transverse if Q n n dS(B) is a submanifold of T*B 
and if, for all x G Q n D dS(B), 

T X (Q W n dS(B)) = T X .(Q W ) n T x (dS'(S)). (2) 

Thus, transversality implies weak tranversality. Some authors use the name clean intersection in- 
stead of weak transversality. These notions of transversality are defined for any pair of submanifolds, 
without being of a symplectic manifold. 

Assuming only clean intersection of Q n and dS(B), then j(Q w H dS(B)) is not, in general, 
an immersed submanifold of T*Q because it can admit multiple points. These multiple points 
x* G )(Q n n dS(B)) are points at which there exist several distinct vector subspaces of the space 
tangent to T*Q in such a way that each one is tangent to a n-dimensional submanifold contained 
in j(Q n n dS(B)). The following result is proved in (T7J Chapter III, Proposition 14.19]. 

Proposition 2.7. Let S : B — > M. be a Morse family. Assume that and dS(B) are weakly 
transverse. Then, the restriction of ) to Q n H dS(B) is of constant rank, and every point of Q n fl 
dS(B) has an open neighborhood V in Q n fl dS(B) whose image j(V) is a Lagrangian submanifold 
of (T*Q,cjq). We say that j(Q n fl dS(B)) is, in a generalized sense, an immersed Lagrangian 
submanifold of (T*Q,uq) which may include multiple points. 

The following proposition determines the relationship between Lagrangian submanifolds and 
Morse families as shown in Remark 12.91 

Proposition 2.8 ( [13] , [E] , [26] ) . Let E be a Lagrangian submanifold of (T*Q, — d#<g). We consider 
the following three properties: 

(i) The closed 1-form induced on E by the Liouville 1-form 9q is exact. 

(ii) There exists a Lagrangian subbundle E of the symplectic bundle (Tx(T*Q),cuq | e ) (that is, 
the restriction to £ of the cotangent bundle {T*Q,ujq)) which is a complement of both TE 
and the restriction of the vertical tangent bundle ofT*Q to E, denoted by V-£,(T*Q). 

(Hi) There exists a surjective submersion it: B — > Q and a Morse family S: U — > R, the latter 
defined on an open subset U of B, such that the Lagrangian immersion generated by S is an 
embedding with image E. 

If properties (i) and (ii) are both satisfied, so is property (Hi). Conversely, if property (Hi) is 
satisfied, then property (i) also holds. 

Remark 2.9. As a direct consequence we deduce that any Lagrangian submanifold is locally, i.e., 
in a neighborhood of each of its points, the image of a Lagrangian immersion generated by a Morse 
family. See [T71 Appendix 7, Proposition 9] for more details. 



2.2 Tulczyjew triple 

The theory of Lagrangian submanifolds gives an intrinsic geometric description of Lagrangian and 
Hamiltonian dynamics [22], [23] • Moreover, it allows us to relate Lagrangian and Hamiltonian 
formalisms using as a main tool the so-called Tulczyjew's triple 

T*TQ ^ — — TT*Q — — ^ T*T*Q . 
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The Tulczyjew map ctQ is an isomorphism between TT*Q and T*TQ. Beside, it is also a symplec- 
tomorphism between these vector bundles considered as symplectic manifolds, i.e. (TT*Q , d^^Q), 
where d^wg is the tangent lift of cuq, and (T*TQ,utq)- For completeness, we recall the construc- 
tion of the symplectomorphism «q. To do this, it is necessary to introduce the canonical involution 
kq on TTQ 

TTQ ^ TTQ 



TTQ 

TQ 



Id 



7 TQ 

TQ, 



defined by 



--odt 



t=o 



d 



s=odt 



i=0 



X(s,t) 



where x '■ ^ 2 ~~ * Q an d X '■ ^ 2 ~ > Q are related by x ( s ; t) — x (t, s). If (q l ) are the local coordinates 
for Q, (q l ,v l ) for TQ and (q l , v l , q 1 , v l ) for TTQ, then the canonical involution is locally given by 

k q (q\ v\ q\ v l ) = (q\ q\ v\ v l ). 

In order to describe a<g is also necessary to define a tangent pairing. Given two manifolds M 
and N, and a pairing (•,•): M x N — >• R between them, the tangent pairing (-, -) T : TM x TA^ — > R 
is determined by 



d_ 

df 



t=0 



«y(t) 



d. 
df 



t=0 



(7 (<),*(*)> 



where 7 : R -> M and 5 : R -> TV. 

Finally, we can define ccq as (ckq (2) , u;) = (z, (w;)) t , where z G TT*Q and to G TTQ. In 
local coordinates (q l ,Pi) for T*Q and (q l , Pi, q l , fit) for TT*Q, we have 



"q {q\PiA\Pi) = (q\q\Pi,Pi) 



The isomorphism /3q : TT*Q — > T*T*Q is just given by /3q = b WQ , where b WQ is the isomorphism 
defined by cjq, that is, t> WQ (-u) = i v ujq. 

The Lagrangian dynamics is described by the Lagrangian submanifold dL (TM) of T*TM where 
L : TM — > R is the Lagrangian function, while the Hamiltonian formalism is described by the La- 
grangian submanifold dH(T*M) of T*TM where H : T*M — > R is the corresponding Hamiltonian 
energy. In this paper we will extend the description of Lagrangian and Hamiltonian dynamics in 
terms of Lagrangian submanifolds and Morse families when controls are involved. Similar studies 
have already been developed by means of Lagrangian submanifolds in the literature for calculus of 
variations [9]. 



2.3 Integrability algorithm 

Let D be an implicit differential equation on a manifold P, that is, a submanifold D of TP. In 
such a case, it is possible to construct an algorithm to extract the integrable part of D in P (see 

Hi). 

A curve 7: / C R — y P is called a solution of a differential equation D if 7(f) C D. An implicit 
differential equation D is said to be integrable at v G D if there is a solution 7: I C R — y P such 
that 7(0) = v. An implicit differential equation D is said to be integrable if it is integrable at each 
point v G D. 
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Proposition 2.10. JTffi Proposition 5] Let Tp : TP — > P be the canonical tangent bundle projection. 
If C = Tp(D) is a submanifold of P and if the mapping 

t p : D 

v i — y Tp(v) 

is a surjective submersion, then the condition D C TC is sufficient for integrability of the implicit 
differential equation D. 

In order to obtain the integrable part of an implicit differential equation we construct the 
following sequence of objects 

(D°,C°,r% (D 1 , C 1 , r 1 ), ... , (D k ,C k ,r k ), ... 

where 

D° = D, C° = C = t p {D), r° = t p , (3) 

D k = D k - 1 nTC k ~ 1 , C k = T P (D k ), r k = T P \ Dk . (4) 

For each k, it is assumed that the sets C k are submanifolds and that the mappings T k are surjective 
submersions. As the dimension of P is finite, the sequence of implicit differential equations D°, 
D 1 , . . . , D k , . . . stabilizes at some index k = s, that is, D s = D 8+1 . The integrable implicit 
differential equation D s C TP is the integrable part of D and it could be empty. 

This integrability algorithm, properly adapted, will be useful in Section \5\ to obtain the ab- 
solutely continuous solutions to optimal control problems which are the most common ones and 
correspond with piecewise constant controls. 

3 Introduction to optimal control problems 

Generally speaking, an optimal control problem from the differential geometric viewpoint is given 
by a vector field depending on parameters called controls, some boundary conditions and a cost 
function whose integral must be minimized or maximized. 

Definition 3.1. An optimal control problem (C, Q, T, L) is given by a control bundle Tc,q : C Q, 
a vector field T defined along the control bundle projection Tc,q, a cost function L: C — > K whose 
functional must be minimized and some endpoint conditions or boundary conditions that must be 
satisfied at initial and/or final time. 

Remember that such a vector field T along tc : q verifies tq o F = tq^q. 



TQ 




Locally, q = T(q, u) and (t q o T) (q, u) = T C , Q (q, u) = q. 

From the optimal control data (C, Q, T, L) we construct the Pontryagin's hamiltonian H : 
T*Q x Q C — >R given by 

H(a q , Ug) = (a q , r(u q )) - L(u q ) (5) 

where u q G C q and a q G T*Q. In coordinates, H(q\pi,u a ) = pjT J \q l ,u a ) — L(q\u a ). This 
Hamiltonian is only valid for the so-called normal optimal solutions. The usual technique to solve 
an optimal control problem is Pontryagin's Maximum Principle [T9] . 
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Theorem 3.2. (Pontryagin's Maximum Principle, PMP) Let [/ C l' and ( r y*,u*): I — > 
C = Q x U be a normal solution of the optimal control problem (C,Q,T,L). Then there exists 
(a*, u*): I -> T*Q x Q C such that: 

(i) it is an integral curve of the Hamiltonian vector field Yh with given initial conditions in the 
states; 

(ii) 7* = ttq o a*, with fiber a*(t) G T**, t sQ where ttq: T*Q — >■ Q is the canonical projection to 
the cotangent bundle; 

(Hi) H{a*{t),u*{t)) — max u gjj _ff(cr*(t), u) almost everywhere, where U is the closure ofU; 
(iv) max^gu H(a*(t), u) is constant everywhere. 



This theorem can be stated without the restriction of having a normal optimal solution. The 
abnormal solutions are associated with the Hamiltonian H(a q ,u q ) = (a q ,T(u q )) that does not 
depend on the cost function. See [21 [6j [TJ [19] for more details on Pontryagin's Maximum Principle 
and optimal control theory. 

This Principle provides us with a set of necessary conditions for optimality. In some optimal 
control problems the time interval is another unknown. If so, then the Hamiltonian function is zero 
along the optimal curve almost everywhere. 

The condition of the maximization over the controls along the optimal solution is usually re- 
placed by a more operative condition on the interior of U, that is, 

^ = 0. (6) 

du a v ' 



4 Morse families and optimal control problems 

In this section Morse families defined in Section 12.11 are considered to describe geometrically the 
optimal control problems from the Hamiltonian viewpoint. 

We first need to identify which is the candidate to be a Morse family in optimal control problems. 
Without doubt the most natural candidate is the Pontryagin's Hamiltonian function H : T*Q Xq 
C — > K defined in (JSJ). According to Definition 12.5} the submersion 7r corresponds with it: T*Q Xq 
C — > T*Q. In adapted local coordinates, we have the following expressions that will be useful in 
the sequel: 

ker Ttt 
(T*Q) W 
dH 

(T*Q) w f]dH(T*Qx Q C) 



_d_ 

du a ( 

(ker Ttt) = (dq\d Pl ), 

^L^qi + ^Ldp + —du a 
dq i dpi % du a 



(q\ Pi , u\ A h B\ C a ) G T*(T*Q Xq C) I Ai 
dH 



dH 
'dq 1 '' 



dpi 



du a 



. 



(7) 



S 



From Definition 12. 51 we know that the Pontryagin's Hamiltonian H : T*Q XqC — > R defines a Morse 
family if and only if the submanifolds (T*Q) n and dH(T*Q XqC) of T*(T*Q XqC) are transverse. 

From Section [2~T1 we obtain the following result. 

Proposition 4.1. Pontryagin's Hamiltonian H : T*Q XqC — > R defines a Morse family if and only 
if the matrix 

dH\ _ ( d 2 H d 2 H 8 2 H 

du ) \dq i du a dpidu a du a du b 

has maximum rank for all (q,p,u) G T*Q Xq C. 



D( ?jP ,„) I — — I 



(<?,P,«) 



Proof. Let us check when the conormal bundle (^Q)^ and dH(T*Q Xq C) are transverse in 
T*(T*Q x Q C), that is, when condition ([I]), 

T a (dH(T*Q x Q C)) + T a (T*Q\ = T a (T*(T*Q x Q C)), 

is satisfied for all a G {T*Q) 1T fl dH(T*Q Xq C). The local generators of the subspaces on the 
left-hand side of the above equation, given in local coordinates (q,p,u, A, B,C), are the following 
ones: 

f d d d d d } 
T a (T*Q)^ = _ _ _ (9) 

m ,„ /m »^ f 9 3 2 H d 2 H d d 2 H 

T a dH(T Q x Q C) = spanj — + 



dq i dq i dq^ dAj dq^dpj dBj dq i du a dC a ' 

d | d 2 H d | g 2 /J 9 

9pi dpidqi dAj dpidu a dCj 

8 d 2 H 8 d 2 H 8 d 2 H 8 



du a du a dqi dAj du a dpj dBj du a du b dC b 

Note that these two tangent spaces satisfy the condition (JTJ) if the matrix 

/ d 2 H d 2 H d 2 H \ 
\dq l du a dpidu a du a du h J 

has maximum rank. Only if this condition is satisfied the Hamiltonian function is a Morse family. 
Otherwise, there is only a strict inclusion T a (dH(T*Q x Q C)) + T a (T*Q) T C T a (T*(T*Q x Q C)).m 



Remark 4.2. In this paper we restrict ourselves to examples where the optimal control problem 
is defined by a Morse family given by the Pontryagin's Hamiltonian H : T*Q x q C — > R. However 
there are many interesting situations where this condition is not fulfilled. For instance, the optimal 
control problem given by the control equation q = u 2 and cost function L = 1 does not satisfy the 
Morse family condition (JSJ for all points (q,p, u) in T*Q Xq C since H(q,p, u) = pu 2 — 1 and 

(dH\ 

Note that H does not define a Morse family at the points (q, 0, 0) because at these points the matrix 
D(q,p,u) does not have maximum rank. In fact, it is also possible to check in this example that 
the intersection of dH(T*Q XqC) and (T*Q) n is not clean at all the points in T*Q XqC. 
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If H : T*Q x Q C -> R defines a Morse family, then E H = dH(T*Q x Q C) n (T*Q) 7r is a 
submanifold of T*(T*Q XqC). The Pontryagin's Hamiltonian is called the generating function for 
the Lagrangian submanifold C H = pr(S H ) of (T*T*Q,cu T * Q ), where pr: T*(T*Q x Q C) -> T*T*Q 
is the natural projection onto the first factor, see Proposition 12.61 Locally, 

OH OH 

£h = {(q,p,P q ,P p ) G T*(T*Q) | 3 u such that (q,u) eC, P q = — (q,p,u),P p = — (q,p,u) 

OH . "I 
a ~0u = ^ I ^ ^ 

) gives the maximum rank in condition OH]), 

there are different kinds of optimal control problems associated with different relative positions of 
Ch with respect to the canonical projection ht*q '■ T*T*Q — > T*Q. 

Definition 4.3. We say that the optimal control problem is regular if the restriction {i^t*q)\c h '■ 
Ch — > T*Q is a local diffeomorphism. 

If the optimal control problem is regular, the submanifold Ch is transverse to the fibers of 

(oh\ 
) 
Ou J 

2 H 

has maximum rank because — — —r has maximum rank. If so, for each (a, p) G T*Q there exists a 

function u* = u*(q,p) by the Implicit Function Theorem defined in a neighbourhood of (q,p) such 
that 

Q,P,^(q,P,u*(q,p)),^{q,p,u*{q,p)),o \ G C H - 

In other words, dH u *(T*Q) = Ch where H u *(q,p) = H(q,p,u*(q,p)). 

If the condition in Definition 14.31 is not fulfilled, then the optimal control problem is singular. 
This singularity may appear because of many different reasons such as 7Tt*q{Ch) $t T*Q or/and 
there are points \i G Ch where 

T^ttq : T^C H — > T a T*Q, a = tt t *q(h) 

is not surjective, that is, \i is a caustic point [31 H]. 

Let us consider now simple examples to show different situations. 
Example 4.4. Consider the optimal control problem given by the control equation q = u and 

L = ^u 3 with u G M. Pontryagin's Hamiltonian H(q,p,u) = pu is obviously a Morse family 

6 

because the matrix 



(0H\ 



has maximum rank. But the optimal control problem is not regular if u = 0. The submanifold Ch 
in (ITT]) is given by 

C H = l(q,p,P g ,P P )eT*(T*R) = R 4 \P q = 0,P p = u,p=lu 2 



2 



>,^A 2 ,0,A)|Ag 
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Observe that the points (x, 0, 0, 0) are caustic and moreover the t^t*q(Ch) = Rx[0, oo) £ M 2 . From 
the local expression of Ch it is also clear that (ttt*q)\c h '- T*Q is a local diffeomorphism as 

long as « / 0. 

Example 4.5 (Singular optimal control problem for LQ systems). Let us consider a general ex- 
ample of singular optimal control problems, see [TTJ [T2J [25] for more details. These problems are 
characterized by the impossibility to determine the controls with first-order necessary conditions 
for optimality. 

Consider the control system 

q(t) = Aq(t) + Bu(t) = r(q,u) 

with cost function 

L(q, u) = -u T Pu + q T Qu + -q T Rq, 

where all matrices A, B, P, Q and R are constant and of suitable size. 

Pontryagin's Hamiltonian is given by H(q,p, u) = p T (Aq + Bu) — L(g, u). As described above, 
the candidate to be a Morse family is the Pontryagin's Hamiltonian. Let 7r: T*Q XqC — > T*Q, the 
local expressions for the elements in equation (|7j) are: 

kerTvr = {(q,p, u, V g , V p , V u ) E T(T*Q x Q C) \ V q = 0, V p = 0} , 

(T*Q)^ = (kerTvr) = {(g,p, M ,A5,C , )GT*(T*gxQC)|C = 0}, 

dH = (p T A -Qu- Rq). dq i + (Aq + Bu) i d Pi + (p T B - Pu - q T Q) a du a , 

(T*Q)^dH(T*Qx Q C) = {(q,p,u,A,B,C) e T*(T*Q x Q C)\Ai= (p T A - Qu - Rq). , 

B i = (Aq + Buf , C a = (p T B - Pu - q T Q) a = 
Let us study the rank of the matrix in condition (jSJ), 

(g ' p ' u) \du a J \dq'du a dp,du a du a du b J V V ; ' 



(12) 



to decide if the Pontryagin's Hamiltonian is a Morse family and if it defines a regular or singular 
optimal control problem. If the above matrix has maximum rank, Pontryagin's Hamiltonian will 
be a Morse family associated with either a regular or singular optimal control problem depending 
on what submatrices of Q12j) have maximum rank. This optimal control problem is regular if P has 
maximum rank. Otherwise, the optimal control problem is singular. 



5 The integrability algorithm for optimal control problems 

Given an optimal control problem we have defined in the previous section the subset Ch i n (TTTT) 
which is an immersed Lagrangian submanifold, as long as the Pontryagin's Hamiltonian defines a 
Morse family. Independently of the manifold structure of Ch, we always have an optimal control 
problem associated with the subset 

pr ((T*Q) n n dH(T*Q x Q C)) = C H C T*T*Q. 

Thus, using the musical isomorphism 

Ht.q : T*T*Q — ► TT*Q 
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defined by the canonical symplectic 2-form ujq on T*Q, we obtain a subset 



V H = %t*q(£h) (13) 

of TT*Q. In this space it is possible to apply the integrability algorithm described in Section 12.31 
Here we will not apply the algorithm, but adapt it to take into account all the peculiarities of an 
optimal control problem. For instance, in an optimal control problem the controls are always mea- 
surable and bounded, and in general piecewise constant. As a result the curves on T*Q satisfying 
the necessary conditions for optimality in Theorem 13.21 are absolutely continuous. Hence, they are 
differentiable almost everywhere and satisfy most of the necessary conditions almost everywhere. 

Unless otherwise stated, in this section the optimal control problems are assumed to be regular. 

Hence, for each point in T*Q, we can describe the control function u in terms of the states and 

dH 

momenta from the equation — — (q,p,u) = 0. 

ou 

A curve o on T*Q XqC satisfies the necessary conditions for optimality in Theorem 13.21 if the 
curve 7 = pr x o a: I C R ->> T*Q, where pr x : T*Q x Q C ->> T*Q, satisfies that 7(f) C §t*q(C h ) 
almost everywhere. 

Note that Vh is a subset of TT*Q with the following local expression 

dH dH 

= {{q,p,V q ,V p ) E T(T*Q) | 3 u such that (q,u) eC, V q = — (q,p,u),V p = - — (q,p,u) 

,9H / 
and —(q,p,u) = 



In order to include the possibility that the solutions are piecewise C l curves we need to modify 
the integrability algorithm described in Section 12.31 

Let tt*q : TT*Q — > T*Q be the canonical tangent bundle projection and consider the subset 
V H Xt t „ q (t>„) V h of the Whitney sum T(T*Q) © T(T*Q). 

A piecewise C 1 curve 7: / C R — y T*Q is called a solution of the optimal control problem 
T^h x t t * q (d h ) T^h if (7~(^) ; 7 + (^)) ^ T^h x t t * q (v h ) T^h for all tel. The implicit differential 
equation determined by Vh x t t * q {t> h ) T^h is said to be integrable at (vi, v 2 ) G T>h x t t * q (v h ) T^h if 
there is a solution 7: I C R — >■ T*Q such that (7~(0), 7 + (0)) = (vi,v 2 ). Note that this notion of 
integrability includes the points of both continuity and discontinuity of the derivative. 

Now we can adapt the integrability algorithm in Section |2~U1 by defining the following sequence 
of objects 

D° = V H x r T , Q {v H ) V H , C° = t t *q{V h x t t , q (v h ) V h ), t° = t t *q\v h x Tt , q(:Dh) v h , 

and 

D k = {(vi, v 2 ) e D k ~ l |3 7:/-)- C k ~ 1 , 7-(0)=Vi, 7 +(0) = t; 2 }, 

C k = r T , Q (D k ), 

r k = r T * Q \ Dk : D k -)• T*Q. 

At some step s the algorithm stabilizes, that is, D s = D s+1 . The integrable implicit differential 
equation D s C T{T*Q) ®T(T*Q) is the integrable part, maybe empty, of V H x t t » q {v h ) T^h- 
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Oq 



Example 5.1. Consider the following optimal control problem 

/■*/ 

minJ = / [u{tf - ifdt, 
J t 

subject to q = u, (JCM, 
= 0, q = 1, 
*/ = 1 > Qf= L 

Pontryagin's Hamiltonian associated with this problem is H(q,p,u) = pu — (u 2 — l) 2 . Obviously, 
this Hamiltonian defines a Morse family because the matrix 

D( q , P ,u) (^) =(01- 12u 2 + 4) 

has maximum rank for all (q,p,u) in T*M Xf C, see Proposition 14.11 

The set T>h of TT*Q in this example has the following local expression: 

{OH OH 
(q,p,Vq,V p ) G T(T*Q) | 3 u such that (q,u) G C, V q = — (q,p,u), V p = — — {q,p,u) 

dH ( \ n 
^(S,^) = 

= {{q,P, V g , V p ) e T(T*Q) | 3 u such that (q,u) eC, V q = u, V p = 0, p - Au{u 2 - 1) = 0} 
= |(x,4m(u 2 — l),u, 0) G T(T*Q) | 3 it such that it is constant} . 

The last equality comes from the fact that p = 0. Hence p is a constant momenta equal to 4u(u 2 — 1). 
Thus the control n must also be constant. 

The starting point of the integrability algorithm is given by 
£° = V H x TTtQ{VH) V H 

= {(xAui(u\ - l),«i,0) G T{T*Q) \uiEU} x Tt * q (v h ) 
{(x,4n 2 (n 2 -l),u 2 ,0) G T(T*Q) \ u 2 e U} , 
C° = T T * Q (V H x TTtQ{VH) V H ) = {(xAui(u\-l))\u 1 eU} , 

r° = t t *q\v h x Tt „ q{ v h) -d h - 

As a consequence of being a fibered product over T*Q, Aui{u\ — 1) = 4:U 2 (ul — 1) so that C° 
is well defined. That condition is fulfilled when U\ = u 2 or if there is a switching time in the 
trajectory where the control changes its value, that is, U\ ^ u 2 . However, if u\ ^ u 2 , the equality 
4mi(m 2 — 1) = 4u 2 (u 2 — 1) is satisfied if and only if ui,u 2 G {—1,0, 1}. Since the trajectories are 
straight lines, the controls can only change from 1 to —1 or vice versa in order to verify the endpoint 
conditions if only one switching time is allowed. For the case u± — u 2 , the endpoint conditions are 
only satisfied if u\ = u 2 = 0. Under the assumption of not having more than one change in the 
control, the next step in the integrability algorithm is the following one: 

D 1 = {((x, 0, u±, 0), (x, 0, u 2 , 0)) G T(T*Q) x TTtQ (T> H ) T(T*Q) \ { Ul = ±1, u 2 = ^1} 
or {ui = u 2 = 0}} , 

C 1 = {(2,0)}, 

r 1 = tt*q\d x '■ D l —> T*Q. 
Here the algorithm stabilizes and the solutions are 
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• 7i(*) = | - 1*) + 1 ^Kt^tf wher6 Ml = ±1 ' M2 = Tl SUch that 7l ^ = 7l+ ^^' 
that is, ititi + 1 = u 2 (ti — tf) + 1. Equivalently, t\ —tf/2. 

• 72 (£) = 1 for ^1 = ^2 = 0. 

If we compute the value of the functional to be minimized along these two families of trajectories, 
we obtain that 

J(7i) = 0, 
J(72) = 1. 

Thus the optimal solution comes from curves like 71. 

6 Towards a Lagrangian formulation of an optimal control 
problem 

After introducing in Section H] the Hamiltonian approach to optimal control problems by Morse 
families, we are going to consider a Lagrangian approach to deal with optimal control problems by 
means of Lagrangian submanifolds of T*(TQ) instead of T*(T*Q). In order to do this we make use 
of the Tulczyjew diffeomorphism which defines a diffeomorphism between T*TQ and T*T*Q, see 
Section 12.21 

Given (C, Q, T, L) an optimal control problem, construct the following subset of T*TQ: 

£c,l = {/i e T*TQ I r> = dL}. (14) 

In optimal control theory there are two different kinds of solutions: the normal ones, that will be 
obtained from £c,l ; and the abnormal ones. The abnormality is characterized by the no dependence 
on the cost function at a first stage. These abnormal curves can be included in our description by 
considering the following subset in T*TQ: 

£ C ,abn = & T*TQ I r> = 0}. 

From now on, we just consider the characterization of normal solutions, but analogous constructions 
can be considered for abnormal solutions. 

In local coordinates (q\q\u a ) for TQ Xq C, the elements in (TT4"]) are written as follows: 

\i = didff + hdq\ 

or or 

r> = adtf + bi— dq 3 +bi— du a , 
oa> ou a 

dL — dq* + — du a 

dq 1 du a 

Hence the elements in £ C L satisfy the following equations: 

• ST ' 9L (15) 



1 3 dq i dq v 

ht du~- = du~-' (16) 

restricted to q 1 = T l (q, u). Condition ffT6|) may introduce additional restrictions in the control space 
C as shown in the following lemma. 
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Lemma 6.1. Define the subset C C C as 

C = {ceC\ dL(V c ) = 0, V V c G kerT c r}, 

then 7T TQ {£ C , L ) C T{C). 

Proof. Let u g G 7Ttq(£c,l), there exists fi Vq in C c ,l such that 

= dL at v g , that is, (fi Vq ,T c r(V c )) = dL(V c ), 

for all V c G T C C such that T(c) = t> g . In particular, for all V c G kerT c r we have that dL(V c ) = 0. 
Hence, c G C and f g G T(C). ■ 

Observe that if T is an immersion, then C = C. 

Assume in the sequel that C is transverse to the fibers of T, that is 

TC + ker Tr = TC. 

Then it is easy to prove the following result. 

Proposition 6.2. Let C = {c G C | dh(V c ) = 0, V V c G kerT c r}, we /jave 

Observe that the restriction of t^q: C — > Q to C is not necessarily the entire space Q- However, 
Proposition 16.21 guarantees we can work indistinctly with and C^ L _. 

Example 6.3. As a toy example consider the optimal control system given by x = u\ + U2 and 
cost function L = x(u\ — 112). In this case, 

£c,l = {( x > u ' a, G M 4 I 3 (ui, u 2 ) G M 2 such that v = u± + u 2 , a = u± — u 2 ,b = x,b = —x} 
= {(0,n,r 2 ,0) I (n,r 2 ) GM 2 } 

On the other hand, C = {(x, uj, u 2 ) G C\x = 0, «i = u 2 }. Note that 7Ptq(£c,l) ^ as proved in 
Lemma 16.11 

Let us consider the Pontryagin's Hamiltonian function H : T*QxqC — > M with local expression 

H(q,p,u) =PiT\q,u) - L(q,u). 
Then the equations in ffT5l) and f|T6|) can be rewritten as follows: 

dq (17) 

restricted to <f = — — (q,b,u) = T l (q,u). Locally, 
opt 

{OH OH OH 

(q\ q\ a h bi) G T*TQ | 3 u such that (q, u) eC, q l = — , a, = — ^— , — = 
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These are the first order necessary conditions for optimality in Pontryagin's Maximum Principle, 
providing that the controls take value in an open set so that the maximization condition of the 
Hamiltonian function over the controls is replaced by the weaker condition (E]). 

By means of the Tulczyjew diffeomorphism 

a Q : TT*Q — > T*TQ 

{q\PiA\Pi) 1 — ► (q % ,Q % ,Pi,Pi), 

and (1T31) it is easy to show that the subsets Ch defined in (fTTj) and £c,l satisfy that 

£c,l = ocq(V h ) = {ocq o $t*q)(£h) ■ 

From Proposition 16.21 we have that £^r L _ = £c,l = olq{T>h)- The above equalities imply the 
following result. 

Proposition 6.4. Let (C, Q, T, L) be an optimal control problem such that the associated Pon- 
tryagin's Hamiltonian is a Morse family, then £c,l is an immersed Lagrangian submanifold of the 
symplectic manifold (T*TQ,utq) ■ 

Let us introduce now the following notion. 
Definition 6.5. The Legendre transformation for an optimal control problem is the mapping 

Leg c , L = t t * q o {a Q l )\ Cc ^ : £ c ,l — ► T*Q. (18) 

It is easy to prove that the optimal control problem is regular if and only if Leg c L is a local 
diffeomorphism, see Definition 14.31 We will say that the optimal control problem is hyperregular if 
Le 

§c l is a global diffeomorphism. 

In the sequel we assume that T,^ : C —> TQ has connected fibers along the points of its image. 
Under this condition we have the following result. 

Lemma 6.6. For any v q in T(C) 

L(c) = L(c) 

for all c and c in r _1 (wg) R C. 

Proof. For v q G T(C) there exists c in C such that T(c) = v q . Since we assume that : C — > TQ 

has connected fibers, for any initial point c in T~ 1 (v q ) C\C there exists a curve 7 : [0, 1] — > C such 
that 7(0) = c, 7(1) = c and T,g{ i y(t)) = v q . Hence 

7(t) E kerT 7(t) r 

From the definition of C we deduce that 

(dL (7(f)), 7(f)) = for all t E [0, 1]. 

Therefore the Lagrangian L is constant along the curve 7, that is, L{^f{t)) = constant. Thus, 
L(c) = L(c) for all c and c in r _1 (w g ) DC ■ 
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Now, it will be obtained the "lagrangian version" of the equations of motion on £c,l for an 
optimal control problem. Any element v G T q Q determines a vertical vector at any point w in the 
fiber over q. That vector lying in T W TQ is called the vertical lift of v at w and it is denoted by v^. 
The vertical lift of v at w is the tangent vector at t = to the curve w + tv. 



v d 
V ™=dt 



(w + tv) . 

t=0 



Having in mind this definition, it is possible to define now the energy function Ec,l '■ £c.l — > K by 
E c ,h{^v q ) = (Hv g , (v q )v q ) - L{c q ), fi Vq G (£c,l)« 9 , c q G C, T(c q ) = v q . (19) 
Lemma 16.61 guarantees that the energy is well-defined. 

The following proposition can be easily proved locally. 
Proposition 6.7. Let H : T*Q Xq C — > R be the Pontryagin's Hamiltonian and c q G C, 

Ec,h(Hv q ) = H(Leg CL (fi Vq ),c q ). 

If the mapping r : C — > TQ is an embedding, then we can select coordinates in such a way that 
the control equations are rewritten as follows: 

q a = u a , 

q a = F a (q\u a ). 

Then the subset Cc,l is locally given by 

dF a dL 

Ul+ ^ (20) 
dF a dL ( J 

a a du a du a 

Note that the submanifold Cc.l can be locally described by the coordinates (q l ,u a ,b a ). 

In this case the Pontryagin's Hamiltonian is H = p a u a +p a F a — L. Note that the condition for 
being a Morse family is trivially satisfied because 

d 2 H \ ( 8 2 H 8 2 H \ ( dT 



dp.idu b J \dp a du b dp a du h J \ du h , 
has maximum rank. 

Due to the local expression of the submanifold £c,l, the energy function in f JT9|) is given by 



E c , L (q\ u\ b a ) = u a — (q\ u a ) - u a b a — (q\ u a ) + b a F a (q\ u a ) - L(q\ u a ). 

The Legendre transformation in the coordinates (q l ,u a ,b a ) for £c,l is written as follows 

(<9L 0F a 
qt 'du^~ ba frF' ba 

Then the Poincare-Cartan 2-form Qc,l on £c,l is defined by 

Q c ,l = (Leg CL )*a;Q 

and the "lagrangian version" of the equations of motion for an optimal control problem is given by 
the following result. 
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Proposition 6.8. The Pontryagin's Hamilton equations are equivalent to the following presymplec- 
tic system 

ix^c,L = dE c ,L- 



6.1 Example: The cheapest stop of a train |2], Section 13.3] 

Let us consider the following control system 

x 1 = x 2 , (21) 
x 2 = u, (22) 

which describes the motion of a train with one control. The aim is to stop the train at a fixed final 
time with a minimum energy. This energy is considered proportional to the integral of the square 
acceleration, that is, we want to find controlled trajectories that maximize the following functional 

1 r T 

L(x 1 (t), x 2 (t), u(t)) dt = - / u 2 {t) dt. 

2 Jo 

Pontryagin's Hamiltonian [19] is given by 

H: T*R 2 x R — > R, 

(x,p,u) i — > pxx 2 + p 2 u - \u 2 . 



Hamilton's equation are given by f l2T|) . (122]) and 

Pi = 0, 

P2 = ~Pl- 

A necessary condition for maximizing the Hamiltonian over the controls is 

dH 



8u »-« = - (23) 



Let us compute the matrix in 



(J* ^ = (a ooi-i) 

\dq t du b dpidu b du a du h J 



Hence, the matrix has maximum rank and the optimal control is regular in the sense of Defini- 
tion 14.31 This example satisfies the typical condition in the literature on the maximum rank of 
d 2 H 

— — — r. In [21 Section 13.31 this problem is solved from there. 
du a du h ^ J 

d 2 H 

However, the above matrix also satisfies that 7: — — r has maximum rank. Thus, by implicit func- 

opidu b 

tion theorem, we can rewrite Hamilton's equations in terms of the local coordinates (x 1 ,x 2 ,pi,u). 
This local approach arises naturally when we consider the Lagrangian formalism for an optimal 
control described in the above section. Note that 

D H = {(x\x 2 ,p 1 ,u,x 2 ,u,0, -pi)} and C c ,l = {(x 1 , x 2 , x 2 ,u,0, -p u p u u)}. 
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The local coordinates for £c,l are (x l ,x 2 ,pi,u) and the set C in Lemma 16.11 is equal to C. The 
condition i^tq{^cx) — in Lemma [6.11 is satisfied with equality. 

The Legendre transformation for this optimal control problem is locally given by 

Leg C;L (z\z 2 ,Pi,M) = (z\s 2 ,pi,u), 
and the energy function is given by 

Ec,l{x 1 i x 2 , p±, u) = H(x 1 ,x 2 ,pi,u,u) = pix 2 + ^u 2 . 
Thus the Poincare-Cartan form on £c,l is given by 

Qc,l = (Leg c l )*ujq = —dpi A dx 1 — du A dx 2 . 

If we write now the "lagrangian version" of the Hamilton's equations for this optimal control 
problem given in Proposition 16.81 ix^c,L = dE c ^, we have 

x 1 = x 2 , p\ = 0, 
x 2 = u, u — —p\. 

6.2 An example of overactuated control system 

Consider now the following control system 

x = Ui, 

y = u 2 + u 3 . 

The cost function whose functional must be minimized is L(x, y, u ly u 2 , u 3 ) = - {u\ + u 2 , + u 2 ). 

The corresponding Pontryagin's Hamiltonian is H(x, y,pi,p 2 , Ui, u 2 , u 3 ) = p\U\ +p 2 (u 2 + u 3 ) — 
- (u\ + u\ + u 2 ). In this example the matrix in (jHJ) is given by 

'0010-10 
1 -1 
,0 1 -1 



because 



dH dH OH 

— =pi- iti, ^— =p 2 - u 2 , —— =p 2 - u 3 . 

OUi ou 2 ou 3 



Obviously, the matrix has maximum rank and the optimal control problem is regular because of 
Definition 14.31 This is an example of overactuated control system, where the controls are not in 
principle repetitive. However, if we work out the equations defining Cc,l, we will see that b\ = Ui, 
b 2 = u 2 = u 3 . Note that 

£c,l = {(x, y, Mi, 2u 2 , 0, 0, Mi, M 2 )}. 

The local coordinates for C c ,l are (x, y, u±, m 2 ) and the set C = {(x, y, Mi, u 2 , m 2 )} C C. Then 
Lemma 16.11 is satisfied with equality 

k T q(£c,l) = {(0, y, Mi, 2u 2 )} = T(C) = {(x, y, u u 2u 2 )}. 
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The Legendre transformation for this optimal control problem is locally given by 

Legc.ii^ Vi u i> u 2 ) = (x, y, u h u 2 ), 
and the energy function is given by 

1 1 

E e ,h(x, y, Mi, u 2 ) = H(x, y, u h u 2 , u 1} u 2 , u 3 ) = u\ + 2uj - - (u\ + 2u 2 2 ) = -u\ + u\. 

Thus the Poincare-Cartan form Qc,l on £c,l is given by 

Q c ,l = (Leg C L )*ujQ = dx A d«i + dy A du 2 . 

If we write now the "lagrangian version" of the Hamilton's equations for this optimal control 
problem given in Proposition 16.81 ^x^c,l = dE c ^, we have 

x = ui, y = 2w 2 , 

iti = 0, ii 2 = 0. 

7 Future work 

As future research line, Morse families might enable a geometric description for the optimal control 
problems defined on manifolds with corners and/or boundaries. Remember that optimal control 
problems are considered a generalization of variational calculus [22] because in the optimal control 
problems the controls take values in a closed and bounded set. The Morse families seem to be a 
useful geometric tool in order to avoid to replace the maximization condition of the Hamiltonian 
over the controls by the weaker condition dH/du = given in (JH]). 

Acknowledgements 

This work has been partially supported by MICINN (Spain) MTM2008-00689, MTM2009-13383, 
MTM2009-08166-E, MTM2010-21186-C02-01 and MTM2010-21186-C02-02; 2009SGR1338 from 
the Catalan government and SOLSUB200801000238 from the Canary Islands government; the 
European project IRSES-project GeoMech-246981 and the ICMAT Severo Ochoa project SEV- 
2011-0087. D. Iglesias acknowledges MICINN for financial support through the Ramon y Cajal 
program 



References 

[1] R. Abraham and J. E. Marsden. Foundations of mechanics. Benjamin/ Cummings Publishing 
Co. Inc. Advanced Book Program, Reading, Mass., 1978. Second edition, revised and enlarged, 
With the assistance of Tudor Ratiu and Richard Cushman. 

[2] A. A. Agrachev and Y. L. Sachkov. Control theory from the geometric viewpoint, volume 87 of 
Encyclopaedia of Mathematical Sciences. Springer- Verlag, Berlin, 2004. Control Theory and 
Optimization, II. 



20 



V. I. Arnold. Singularities of caustics and wave fronts, volume 62 of Mathematics and its 
Applications (Soviet Series). Kluwer Academic Publishers Group, Dordrecht, 1990. 

V. I. Arnold, S. M. Gusein-Zade, and A. N. Varchenko. Singularities of differentiable maps. 
Volume 1. Modern Birkhauser Classics. Birkhauser/Springer, New York, 2012. Classification 
of critical points, caustics and wave fronts, Translated from the Russian by Ian Porteous based 
on a previous translation by Mark Reynolds, Reprint of the 1985 edition. 

M. Barbero-Linan and M. C. Munoz-Lecanda. Constraint algorithm for extremals in optimal 
control problems. Int. J. Geom. Methods Mod. Phys., 6(7):1221-1233, 2009. 

M. Barbero-Linan and M. C. Munoz-Lecanda. Geometric approach to Pontryagin's maximum 
principle. Acta Appl. Math., 108(2) :429-485, 2009. 

A. Bressan and B. Piccoli. Introduction to the mathematical theory of control, volume 2 of 
AIMS Series on Applied Mathematics. American Institute of Mathematical Sciences (AIMS), 
Springfield, MO, 2007. 

F. Bullo and A. D. Lewis. Supplementary chapters of geometric Control of Mechanical Systems, 
http:/ /penelope.mast.queensu.ca/smcs/, 2005. 

M. de Leon, F. Jimenez, and D. Martin de Diego. Hamiltonian dynamics and constrained 
variational calculus: continuous and discrete settings. arXiv:1108.5570 [math-ph], 2012. 

M. de Leon and P. R. Rodrigues. Methods of differential geometry in analytical mechanics, vol- 
ume 158 of North-Holland Mathematics Studies. North-Holland Publishing Co., Amsterdam, 
1989. 

M. Delgado-Tellez and A. Ibort. On the geometry and topology of singular optimal control 
problems and their solutions. Discrete Contin. Dyn. Syst., (suppl.):223-233, 2003. Dynamical 
systems and differential equations (Wilmington, NC, 2002). 

M. Delgado-Tellez and A. Ibort. A numerical algorithm for singular optimal LQ control 
systems. Numer. Algorithms, 51(4):477-500, 2009. 

K. Grabowska and J. Grabowski. Variational calculus with constraints on general algebroids. 
J. Phys. A, 41(17):175204, 25, 2008. 

V. Guillemin and S. Sternberg. Geometric asymptotics. American Mathematical Society, 
Providence, R.I., 1977. Mathematical Surveys, No. 14. 

L. Hormander. Fourier integral operators. I. Acta Math., 127(l-2):79-183, 1971. 

V. Jurdjevic. Geometric Control Theory, volume 52 of Cambridge Studies in Advanced Math- 
ematics. Cambridge University Press, Cambridge, 1997. 

P. Libermann and C.-M. Marie. Symplectic geometry and analytical mechanics, volume 35 of 
Mathematics and its Applications. D. Reidel Publishing Co., Dordrecht, 1987. Translated from 
the French by Bertram Eugene Schwarzbach. 

G. Mendella, G. Marmo, and W. M. Tulczyjew. Integrability of implicit differential equations. 
J. Phys. A, 28(1):149-163, 1995. 



21 



[19] L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko. The mathe- 
matical theory of optimal processes. Translated from the Russian by K. N. Trirogoff; edited by 
L. W. Neustadt. Interscience Publishers John Wiley & Sons, Inc. New York-London, 1962. 

[20] H. J. Sussmann. An introduction to the coordinate-free maximum principle. In Geometry 
of Feedback and Optimal Control, volume 207 of Monogr. Textbooks Pure Appl. Math., page 
463-557. Dekker, New York, 1998. 

[21] H. J. Sussmann. Geometry and optimal control. In Mathematical Control Theory, page 
140-198. Springer, New York, 1999. 

[22] H. J. Sussmann and J. C. Willems. 300 years of optimal control: From the brachystochrone 
to the maximum principle. IEEE Control Syst., 1997. Historical Perspectives. 

[23] W. M. Tulczyjew. Les sous-varietes lagrangiennes et la dynamique hamiltonienne. C. R. Acad. 
Set. Pans Ser. A-B, 283(8):Av, 15-18, 1976. 

[24] W. M. Tulczyjew. Les sous-varietes lagrangiennes et la dynamique lagrangienne. C. R. Acad. 
Set. Pans Ser. A-B, 283(8):Av, A675-A678, 1976. 

[25] K. Volckaert and D. Aeyels. The Gotay-Nester algorithm in singular optimal control. In 

Proceedings of the 38th IEEE Conference on Decision and Control, volume 1, page 873-874, 
1999. 

[26] A. Weinstein. Symplectic manifolds and their Lagrangian submanifolds. Advances in Math., 
6:329-346, 1971. 



22 



